Are Many Reactive Agents Better Than a Few Deliberative Ones?

Problem solvers fall along a wide spectrum ranging from highly deliberative to highly re-active. Highly deliberative systems are able to design optimally efficient solutions to problems , but they require complete world models and consume inordinate computational resources. Reactive systems move in real time but cannot guarantee efficient solutions. They are also subject to looping behavior. One way to generate incrementally more efficient solutions is to be incrementally more deliberative, e.g., to increase the amount of mental search between actions. This paper presents an alternative method for generating more efficient solutions: increasing the number of reactive agents simultaneously attacking a given problem. This method provides a second, orthogonal degree of freedom. We find that in many domains, increasing agents is dramatically superior to increasing single-agent deliberative-ness. This is because solution quality improves rapidly as more reactive agents are added, but search time only increases linearly. This contrasts with adding more deliberativeness, which incurs exponentially increasing time costs. Ample empirical evidence is presented to support our conclusions. 1 Introduction This paper considers two aspects of computational problem solving: (1) search time-how long it takes to come up with a solution. (2) solution quality—how good that solution is, in terms of resources needed to execute it. There is an intuitive trade-off between (1) and (2). The longer we think about a problem, the better chance we have of finding a good solution. While search algorithms like A* [Hart et al., 1968] strive to limit (1) while optimizing (2), time limitations often force us to settle for suboptimal, or "satisficing" [Simon, 1957], solutions. Different situations will place different emphases on (1) and (2). Consider the problem of sending an inter-planetary probe to Neptune. In this case, it may be worth spending days or weeks to plot an optimal trajec-tory, since such calculations could save months of travel time. On the other hand, consider the case of Hernan Cortes, the Spanish conqueror of Mexico. While still a teenager in Spain, finding himself on the wrong end of a jealous husband's musket, Cortes immediately devised a plan to travel to the New World. The efficiency of his plan was not critical. What was important was that he get started right away. This paper studies search time versus solution quality in the context of the Real-Time-A* (RTA*) algorithm devised by [Korf, 1990]. The next section reviews how RTA* interleaves planning and execution, and how this leads …